[Infra] Change host count KPI query #188950

jennypavlova · 2024-07-23T14:28:44Z

Summary

This PR adds an endpoint to get the hosts (monitored by the system integration) count. Currently, it supports only hosts but it can be extended to other asset types (if/when needed). So the endpoint ( POST /api/infra/{assetType}/count ) supports only 'host' as {assetType}.

⚠️ This PR adds only the endpoint - it is not used yet! To avoid having different host counts and results shown on the UI this PR is not updating the hook responsible for the request because currently the hosts shown in the table are not filtered by the system integration (showing the filtered result of this endpoint can lead to inconsistencies between the count and the results shown in the table) Once #188756 and #188752 are done we can use this endpoint.

Testing

It can't be tested in the UI so we can test it:

Using curl:

curl --location -u elastic:changeme 'http://0.0.0.0:5601/ftw/api/infra/host/count' \
--header 'kbn-xsrf: xxxx' \
--header 'Content-Type: application/json' \
--data '{
   "query": {
      "bool": {
         "must": [],
         "filter": [],
         "should": [],
         "must_not": []
      }
   },
   "from": "2024-07-23T11:34:11.640Z",
   "to": "2024-07-23T11:49:11.640Z",
   "sourceId": "default"
}'

In case of testing with oblt replace the elastic:changeme with your user and password

obltmachine · 2024-07-23T14:28:58Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-07-24T11:41:21Z

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

crespocarlos

Thanks for this change Jenny. I had a quick look but haven't had the chance to test it yet.

crespocarlos · 2024-07-24T12:02:46Z

x-pack/plugins/observability_solution/infra/server/routes/infra/lib/host/get_hosts_count.ts

+                  {
+                    term: {
+                      'event.module': 'system',
+                    },
+                  },
+                  {
+                    term: {
+                      'metricset.module': 'system',
+                    },
+                  },


We could use termQuery helper functions. Also, how about creating constants for the field names?

Suggested change

{

term: {

'event.module': 'system',

},

},

{

term: {

'metricset.module': 'system',

},

},

...termQuery(EVENT_MODULE, 'system')

...termQuery(METRICSET_MODULE, 'system'),

crespocarlos · 2024-07-24T12:04:30Z

x-pack/plugins/observability_solution/infra/server/routes/infra/lib/host/get_hosts_count.ts

+          ...queryBool,
+          filter: [
+            ...queryFilter,
+            ...rangeQuery,


We could use the rangeQuery helper function here.

Suggested change

...rangeQuery,

...rangeQuery(new Date(from).getTime(), new Date(to).getTime())

crespocarlos · 2024-07-24T12:06:11Z

x-pack/plugins/observability_solution/infra/server/routes/infra/index.ts

+
+  framework.registerRoute(
+    {
+      method: 'post',


I wonder if we this could be a GET request.

I need to pass the query, filters, time range, etc. so we can even expand more the body if it is a POST. I am not sure how much will those params extend in the future so I was thinking that having the consistency with the other endpoints with similar functions can help here. But I don't have a strong opinion on that. Do you think it's better to have query params instead of the request body and convert to GET?

GET operation is preferable in this case, since nothing is being created. Infra uses POST for historical reasons, but most of them should be GET instead. The complicated part is on how to pass the filters in the query params.

As discussed I created an issue to investigate the option to use GET instead of POST in our APIs #189170 There are some benefits and concerns described there and it would be nice also to keep them consistent and have GET requests where possible ( I totally agree that it will be better but we still need to check if the url limitations affect us) So this change won't be part of this PR

crespocarlos · 2024-07-25T11:08:37Z

x-pack/plugins/observability_solution/infra/server/routes/infra/lib/constants.ts

+export const EVENT_MODULE = 'event.module';
+export const METRICSET_MODULE = 'metricset.module';


Perhaps these could be in the constants in common instead. It will allow the client to also access them.

Good point, thanks ✅

crespocarlos

LGMT! Thanks for this change.

kibana-ci · 2024-07-25T13:14:05Z

💚 Build Succeeded

Buildkite Build
Commit: befee95
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-188950-befee9548c6f
Observability Deployment

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`infra`	1576	1577	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`infra`	1.5MB	1.5MB	+540.0B

History

💛 Build #223508 was flaky 9063842
💔 Build #223456 failed 9c8e4af

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jennypavlova

Add endpoint that returns the hosts count

2af9b05

jennypavlova added the Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team label Jul 23, 2024

jennypavlova self-assigned this Jul 23, 2024

jennypavlova added 8 commits July 23, 2024 17:52

Rename host to asset type and add param

8f5822d

Update test

1d89159

Move and rename endpoint to asset count and add params validation

423f5fb

Rename to asset count

0078b7c

Change quert types

d093d69

Remove unused kquery filter

d2e537c

Update README

c3e0d2c

Update README

03b1db2

jennypavlova added release_note:skip Skip the PR/issue when compiling release notes backport:skip This commit does not require backporting labels Jul 24, 2024

Merge branch 'main' into 188757-infra-change-host-count-kpi-query

9c8e4af

jennypavlova marked this pull request as ready for review July 24, 2024 11:41

jennypavlova requested review from a team as code owners July 24, 2024 11:41

crespocarlos reviewed Jul 24, 2024

View reviewed changes

botelastic bot added the ci:project-deploy-observability Create an Observability project label Jul 24, 2024

jennypavlova added 2 commits July 24, 2024 15:53

CR comments

1375ddb

Merge branch 'main' into 188757-infra-change-host-count-kpi-query

9063842

jennypavlova requested a review from crespocarlos July 24, 2024 15:43

crespocarlos reviewed Jul 25, 2024

View reviewed changes

Move constants

befee95

crespocarlos approved these changes Jul 25, 2024

View reviewed changes

crespocarlos mentioned this pull request Jul 25, 2024

[Infra] Consider only docs from system and APM in the hosts view #188756

Closed

jennypavlova enabled auto-merge (squash) July 25, 2024 12:52

jennypavlova merged commit fd8b7f6 into elastic:main Jul 25, 2024
22 checks passed

kibanamachine added the v8.16.0 label Jul 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Infra] Change host count KPI query #188950

[Infra] Change host count KPI query #188950

jennypavlova commented Jul 23, 2024 •

edited

Loading

obltmachine commented Jul 23, 2024

elasticmachine commented Jul 24, 2024

crespocarlos left a comment

crespocarlos Jul 24, 2024

crespocarlos Jul 24, 2024

crespocarlos Jul 24, 2024

jennypavlova Jul 24, 2024

crespocarlos Jul 24, 2024

jennypavlova Jul 25, 2024

crespocarlos Jul 25, 2024

jennypavlova Jul 25, 2024

crespocarlos left a comment

kibana-ci commented Jul 25, 2024 •

edited

Loading

	...rangeQuery,
	...rangeQuery(new Date(from).getTime(), new Date(to).getTime())

		export const EVENT_MODULE = 'event.module';
		export const METRICSET_MODULE = 'metricset.module';

[Infra] Change host count KPI query #188950

[Infra] Change host count KPI query #188950

Conversation

jennypavlova commented Jul 23, 2024 • edited Loading

Summary

Testing

obltmachine commented Jul 23, 2024

🤖 GitHub comments

elasticmachine commented Jul 24, 2024

crespocarlos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

crespocarlos left a comment

Choose a reason for hiding this comment

kibana-ci commented Jul 25, 2024 • edited Loading

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

History

jennypavlova commented Jul 23, 2024 •

edited

Loading

kibana-ci commented Jul 25, 2024 •

edited

Loading